The working set stops being a blob every submit overwrites (v0.5.0) - #109

C4 is the symptom. This is the cause.

That target carries a stale NEXT: run the sweep directive, a note about attempt 7's formatting failure, and a salvage fragment from a 600-second timeout — all pinned in the frontier with no way to shift them except an admin editing the field by hand. None of the three causes is specific to C4.

1. Replacement → merge

UPDATE targets SET state = $2. Last writer wins over the whole blob, so to add one key an agent had to reconstruct the entire working set. Two consequences: any submit could destroy what earlier work established, and any key an agent didn't recognise got copied forward forever — dropping it was indistinguishable from destroying a real result.

State now merges per key. Omission changes nothing. $retract: ["key"] is the only way to remove one, which turns retiring a dead end into a deliberate, attributable act an agent can perform itself. That is the part that answers "I can't be fixing fields for the rest of our lives."

2. Loss → append-only facts

The 64 KB cap truncated by keeping the tail, so the first thing dropped as state grew was the accumulated head: the established facts.

Facts move to target_facts (migration 015) — append-only rows, deduped by claim so replays and independent rediscovery collapse to one, each carrying the contribution that established it, retracted rather than deleted when superseded (same principle as the ledger), and outside the blob cap entirely.

3. Staleness → derived next steps

NEXT: … was hand-written prose describing work, so it went stale the moment that work landed. next_steps is now derived from the task graph on every read: claimable tasks cheapest-first, counts in flight and expired, and a stalled flag. Nothing to keep up to date because nothing is stored.

The expired count is a bonus — it surfaces the systematic-failure signal (C4's pile of 15 timed-out tasks) that no prose field would ever have shown.

Two bugs this surfaced

• Truncation tombstones. A {truncated, note, tail} marker is written by the platform, not built by an agent, so the next real update replaces it rather than merging. Merging would have pinned those three keys permanently — a fresh species of the exact debris this change removes. Caught by an existing test.
• The salvage was the source. salvageTimedOutRun spread the prior state back over itself to avoid clobbering it. That worked, but re-published every key the run never touched — literally how C4's timeout_salvage outlived its attempt. It now sends only its own key and lets the server merge.

Compatibility

The executor's output schema now documents facts and $retract, so the mechanism doesn't ship inert. Old runners stay compatible: they send the whole blob, and merging that is idempotent. Server-side merge deploys on merge to main, ahead of any CLI release that depends on it. Existing free-form states are untouched and keep rendering.

v0.5.0 — unlike #107/#108, src/executor.ts is in the CLI bundle (verified: the new schema text appears in dist/givework.mjs), so this changes the published runner and not just the Worker.

Testing

587 tests, 14 new. Verified end to end on a C4-shaped target:

── start: legacy blob with stale directive + debris
   state keys : phase2, attempt7_note
   facts      : (none)
── after agent A adds a fact (never re-sent the blob)
   state keys : phase2, attempt7_note
   facts      : δn < 0 for all n ≤ 10^6 … vacuous below n ≈ 1.6e34
── after agent B retracts the debris — no admin involved
   state keys : cursor
   facts      : δn < 0 … (survived)
── after a timeout salvage
   state keys : cursor, timeout_salvage
   facts      : δn < 0 … (survived)

Not in this PR

• Surfacing facts / next_steps on the conjecture page. Server-side only here; a migration plus a submit-path change deserves review without UI noise.
• The informativeness guard. Nothing yet stops a task whose success criterion is trivially satisfiable (C4's actual mathematical failure, as distinct from its state mess). That wants postconditions on tasks validated at decomposition time, and is its own design.
• C4's own data. No production rows touched. Once this ships, the next agent to work that target can retract the debris itself.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QQgFEPRY4W74D4eqawQ4A6